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Maximally Stabilizing Task Release Control 
Policy for a Dynamical Queue 

Ketan Savla Emilio Frazzoli 
Abstract 

In this paper, we introduce a model of dynamical queue, in which the service time depends on the 
server utilization history. The proposed queueing model is motivated by widely accepted empirical laws 
describing human performance as a function of mental arousal. The objective of this paper is to design 
task release control policies that can stabilize the queue for the maximum possible arrival rate, assuming 
deterministic arrivals. First, we prove an upper bound on the maximum possible stabilizable arrival rate 
for any task release control policy. Then, we propose a simple threshold policy that releases a task to 
the server only if its state is below a certain fixed value. Finally, we prove that this task release control 
policy ensures stability of the queue for the maximum possible arrival rate. 

I. Introduction 

In this paper, we introduce a novel model for a dynamical queue, in which service times depend on the 
utilization history of the server. In other words, we consider the server as a dynamical system, and model 
the service time as a function of its state. Given this model, we consider the case in which new tasks 
arrive at a deterministic rate, and propose a task release control architecture that schedules the beginning 
of service of each task after its arrival. We propose a simple threshold policy that releases a task to the 
server only if its state is below a certain fixed value. The proposed task release control policy is proven 
to ensure stability of the queue for the maximum possible arrival rate, where the queue is said to be 
stable if the number of tasks awaiting service does not grow unbounded over time. 

Queueing theory is a framework to study systems with waiting lines. An extensive treatment of queueing 
systems can be found in several texts, e.g., see [2], [3]. The queueing system considered in this paper 
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falls in the category of queueing systems with state-dependent parameters, e.g., see [4]. In particular, 
we consider a queueing system with state-dependent service times. Such systems are useful models 
for many practical situations, especially when the server corresponds to a human operator in a broad 
range of settings including, for example, human operators supervising Unmanned Aerial Vehicles, and 
job floor personnel in a typical production system. The model for state-dependent service times in this 
paper is inspired by a well known empirical law from psychology — the Yerkes-Dodson law [5] — which 
states that human performance increases with mental arousal up to a point and decreases thereafter. Our 
model in this paper is in the same spirit as the one in [6], where the authors consider a state-dependent 
queueing system whose service rate is first increasing and then decreasing as a function of the amount 
of outstanding work. However, our model differs in the sense that the service times are related to the 
utilization history rather than the outstanding amount of work. A similar model has also been reported 
in the human factors literature, e.g., see [7]. 

The control architecture considered in this paper falls under the category of task release control, which 
has been typically used in production planning to control the release of jobs to a production system in 
order to deal with machine failures, input fluctuations and variations in operator workload (see, e.g., [8], 
[9]). The task release control architecture is different from an admission control architecture, e.g., see 
[10], [6], where the objective is, given a measure of the quality of service to be optimized, to determine 
criteria on the basis of which to accept or reject incoming tasks. In the setting of this paper, no task 
is dropped and the task release controller simply acts like a switch regulating access to the server and 
hence effectively determines the schedule for the beginning of service of each task after its arrival. 

The contributions of the paper are threefold. First, we propose a novel dynamical queue, whose server 
characteristics are inspired by empirical laws relating human performance to utilization history. Second, 
we provide an upper bound on the maximum possible stabilizable arrival rate for any task release 
control policy by postulating a notion of one-task equilibrium for the dynamical queue and exploiting its 
optimality. Third, we propose a simple threshold policy that matches this bound, thereby also giving the 
stability condition for this queue. 

II. Problem Formulation 

Consider the following deterministic single-server queue model. Tasks arrive periodically, at rate A, 
i.e., a new task arrives every 1/A time units. The tasks are identical and independent of each other and 
need to be serviced in the order of their arrival. We next state the dynamical model for the server, which 
determines the service times for each task. 
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A. Server Model 

Let x(t) be the server state at time t, and let b : M — > {0, 1} be such that b(t) is 1 if the server is busy 
at time t, and otherwise. The evolution of x(t) is governed by a simple first-order model: 



where r is a time constant that determines the extent to which past utilization affects the current state of 
the server, and xq £ [0, 1] is the initial condition. Note that the set [0, 1] is invariant under the dynamics 



The service times are related to the state x(t) through a map S : [0, 1] — > M>o- If a task is allocated to 
the server at state x, then the service time rendered by the server on that task is S(x). Since the controller 
cannot interfere the server while it is servicing a task, the only way in which it can control the server state 
is by scheduling the beginning of service of tasks after their arrival. Such controllers are called task release 
controllers and will be formally characterized later on. In this paper we assume that: S(x) is positive 
valued, continuous and convex. Let 5 m i n := min{5(x) | x G [0, 1]}, and 5 max := max{<S(0),<S(l)}. 

The solution to Equation ([T]) is x(t) = e~ l l T ( J* ^b{s)e s / T ds + xq^J . This implies that the server state 
x(t) is increasing at times when the server is busy, i.e, when b(t) = 1, and decreasing at times when 
the server is not busy, i.e., when b(t) = 0. Note that S(x) is not necessarily monotonically increasing in 
x, since it has been noted in the human factors literature (e.g., see [5]) that, for certain cognitive tasks 
demanding persistence, the performance (which in our case would correspond to the inverse of S(x)) 
could increase with the state x when x is small. This is mainly because a certain minimum level of 
mental arousal is required for good performance. An experimental justification of this server model in 
the context of humans-in-loop systems is included in our earlier work [1], where S(x) for that setup was 
found to have a U-shaped profile. 

B. Task Release Control Policy 

We now describe task release control policies for the dynamical queue. Without explicitly specifying 
its domain, a task release controller u acts like an on-off switch at the entrance of the queue. Therefore, 
in short, u is a task release control policy if u(t) E {ON, OFF} for all t > 0, and an outstanding task 
is assigned to the server if and only if the server is idle, i.e., when it is not servicing a task, and when 
u = ON. Let U be the set of all such task release control policies. Note that we allow U to be quite 
general in the sense that it includes control policies that are functions of A, S, x, etc. 

July 22, 2010 DRAFT 



X{t) = (b(t) - X{t)) /T, 



x(0) = x 



(1) 




4 



C. Problem Statement 

We now formally state the problem. For a given r > 0, let n u (t, r, A, Xq, uq) be the queue length, i.e., 
the number of outstanding tasks, at time t, under task release control policy u 6W, when the task arrival 
rate is A and the server state and the queue length at time t = are xq and no respectively. Define the 
maximum stabilizable arrival rate for policy u as: 

Amax^, u) = SUp < A | lim Slip 1l u (t, T, A, Xq, Uq) < +OO, \/xq G [0, 1] , V«o S N > . 

[ <->+oo J 

The maximum stabilizable arrival rate over all policies is defined as A^ ax (r) = sup ugW A max (r, u). 
A task release control policy u is called maximally stabilizing if, for any xo £ [0, 1], no 6 N, r > 0, 
limsup < _ >+00 n u (t, r, A, xo, no) < +oo for all A < A^ ax (r), The objective in this paper is to design a 
maximally stabilizing task release control policy for the dynamical queue whose server state evolves 
according to Equation ([1]), and whose service time function S(x) is positive, continuous and convex. 

III. Upper Bound 

In this section, we prove an upper bound on A^ ax (r). We do this in several steps. We start by introducing 
a notion of one-task equilibrium for the dynamical queue under consideration. 

A. One-task Equilibrium 

Let Xi be the server state at the beginning of service of the i-th task and let the queue length be zero 
at that instant. The server state upon the arrival of the {i + l)-th task is then obtained by integration of 
([T]) over the time period [0, 1/A], with initial condition xo = Xj. Let x\ denote the server state when it 
has completed service of the i-th task. Then, x\ = 1 — (1 — Xi)e~ s ( x '^ T . Assuming that S(xi) < 1/A, 
we get that x i+1 = ^-(i/A-sOrO)/^ and fi nSL ii y x . +1 _ (f _ (f _ x ^ e -S{x i )/r^ e {S{x i )-x/\)/T = 

(x{ — 1 + e s ( Xi ^ T ) . If A and r are such that x- l+ \ = X{, then under the trivial control policy u(t) = 
ON, the server state at the beginning of all the tasks after and including the i-th task will be X{. We then say 
that the server is at one-task equilibrium at Xj. Therefore, for a given A and r, the one-task equilibrium 
server states correspond to x € [0,1] that satisfy x = (x — 1 + e s ^ x ^ T ) e~ and S(x) < 1/A, i.e., 
S(x) = rlog (l - (1 - e^)x) and S(x) < 1/A. Let us define a map TZ : [0, l]xl + xl + 4 R+ as: 

K(x,t, A) := rlog (l - (1 - e^)x\ . (2) 

The following result establishes a key property of lZ(x, r, A). 

Lemma 3.1: For any r > and A > 0, the function 1Z defined in Equation ([2]) is strictly concave in 
x, and ^K(x, r, A) > for all x G [0, 1]. 
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Proof: Taking the first and second partial derivatives of Equation ([2]) with respect to x, we get that, 



K{x, t, A) = r— , —^K(x, t, A) 



dx ' ' l-(l-eX?)a' dx2 ' ' [1 - (1 - e^)x} 2 ' 
These expressions show that, for a given r > and A > 0, ^TZ(x, r, A) < for all x £ M. Therefore, 1Z 
is strictly concave in x. Also, J^lZ(x, r, A)| 1= i = r ^1 — e~ > for all r > and A > 0. Therefore, 
by the concavity of 11, ^K(x, r, A) > for all x £ [0, 1]. ■ 
For a given r > and A > 0, define the set of one-task equilibrium server states as: 

x cq (r, A) := {x e [0, 1] | S(x) = TZ(x, r, A)} . (3) 

Remark 3.2: Note that we did not include the constraint S(x) < 1/A in the definition of x eq (r, A) 
in Equation ([3]). This is because this constraint can be shown to be redundant as follows. Equation ([2]) 
and Lemma |3.1| imply that, for any r > and A > 0, lZ(x, r, A) is strictly increasing in x and hence 
n(x,r,\) < 72(1, r, A) = 1/A for all x e [0,1]. Therefore, cS(x eq (r,A)) = K(x eq (T, A), r, A) < 1/A. 

We introduce a couple of more definitions. For a given r > 0, let 

A™ x (r) :=max{A > | x eq (r, A) + 0} , 

(4) 

xth(r) :=x eq (r,A^(r)). 

We now argue that the definitions in Equation ((4]) are well posed. Consider the function Six) — 
K(x, t, A). Since 71(0, r, A) = for any r > and A > 0, and S(0) > 0, we have that 5(0) -K(0, r, A) > 
for any r > and A > 0. Since K(l, r, A) = 1/A, 5(1) -TZ{1, r, A) < for all A < l/5 max . Therefore, 
by the continuity of S(x) — 1Z(x,t,\), the set of equilibrium server states, as defined in Equation ([3]), 
is not-empty for all A < l/5 max . Moreover, since 1Z(x,t,\) < 72.(1, r, A) = 1/A for all x E [0,1], 
S(x) — 1Z(x, t, A) > S(x) — 1/A for all x G [0, 1]. Therefore, for all A > l/«S m i n , the set of equilibrium 
states, as defined in Equation ([3]), is empty. Hence, A™ ax ("r) and x t h(r) are well defined. In general, for 
a given r > and A > 0, x eq (x,r) is not a singleton, e.g., see Figure [T] However, due to the strict 
convexity of S(x) — 1Z(x, r, A) in x as implied by Lemma 3.1 £th( r ) contains only one element. In the 
rest of the paper, x t h(r) will denote this single element. 

In the rest of the paper, we will restrict our attention on those r and S(x) for which xth_{r) < 1. 
Loosely speaking, this is satisfied when S{x) is increasing on some interval in [0, 1] and the increasing 
part is steep enough (e.g., see Figure [T}. It is reasonable to expect this assumption to be satisfied in 
the context of human operators whose performance deteriorates quickly at very high utilizations. The 
implications of the case when x^r) = 1 are discussed briefly at appropriate places. 

The following property of S(x) will be used later on. 
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Fig. 1. A typical S(x) along with TZ(x,t,\) for three values of A: Ai ow , A rae d and A™ ax (r) in the increasing order. Also, 
Xeq (r, Aiow) = {.-cic„}, x cq (t, A rac d) = {^mcd.i , Smcd.2 } and x eq (t, A™"" (t)) = {sth(r)}. Note that, since x t h(r) < 1, A™ ax (r) 
is the value of A at which TZ(x, r, A) is tangent to S(x). 



Lemma 3.3: For any r > 0, if Xth( r ) < 1, then ^S(x)\ x=Xa ^ T - ) > 0. 

Proof: The convexity of S along with strict concavity of 1Z from Lemma 3.1 imply that S(x) — 
TZ{x, r, A) is strictly convex in x for any r > and A > 0. Therefore, by the definition of xth{i~) and 
A™ 3x (t), if xth(T) < 1 then Xth(r) corresponds to the unique minimum of S(x) — 1Z (x, r, A™ ax (r)). 
Hence, (5(x) — 7£ (x, r, A™ ax (r))) | x=:rth ( T ) = for any r > 0. The result follows by combining this 
with the fact that -j^TZ (x, r, A™ ax (r)) \ x = Xtll M > f° r an y r > from Lemma 



3.1 



Remark 3.4: The proof of Lemma 3.3 implies that, if x^t) < 1 then (<S(a?) — 7£ (x, r, A™ ax (r))) > 
for all x G (xth(T), 1] and hence S(x) — 1Z (x,r, A"q ax (r)) > for all x G (x t h(T), 1]. In particular, 
5(1) > TZ (l,T, A™ ax (r)) = Am i( r ) > i-e., A™ ax (r) (which will be proven to be the maximum stabilizable 
arrival rate) is strictly greater than 1/5(1), which is the rate at which the server is able to service tasks 
starting with the initial condition xq = 1 and servicing tasks continuously thereafter. 
We next consider a static problem and establish results there that will be useful for the dynamic case. 

B. The Static Problem 

Consider the following n-task static problem: Given n tasks, what is the fastest way for the dynamical 
server to service these tasks starting with an initial state x and ending at final state x. We emphasize here 
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that all the n tasks are initially enqueued and no new tasks arrive. Let Tf(x, r, n, u) be the time required 
by the task release control policy u G U for the n-task static problem with initial and final server state 
x E [0, 1]. We first provide a result that relates the time for the one-task static problem to A™ ax (r). 
Lemma 3.5: For any x £ [0, 1], r > and u £U, we have that Tf(x, r, 1, u) > l/A™ ax (Y). 

Proof: First consider the policy u that assigns the task to the server right away. In this case, 
Tf(x,r, l,u) is the sum of S(x) and the idle time to allow the server state to return to x. From the 
definition of one-task equilibrium server states, it follows that X7(x,t, l,u) is such that, 

x e x eq (t, ) . (5) 

In other words, Tt (x, r, 1, u) is the inverse of the arrival rate such that if the server starts at state x with 
zero queue length, then the server will be able to service a task and get back to state x exactly at the instant 
when the next task arrives. Now, consider a policy u x - that waits for some initial time until the server state 
reaches state x~ before assigning the task to the server. By definition, u = u x . In this case, also referring 
to Figure |2j Ty (x, r, 1, u x - ) is the sum of initial idle time t~ for the server to reach state x~ , the service 
time S(x~) and the idle time t + for the server state to return to x. Note that only those u x - are feasible 
for which the server state after service time S{x~) is not less than x. From the rearrangement argument 
illustrated in Figure [2j it can be inferred that, for any such u x -, Tf(x,r,l,u x -) = Tf(x~,r,l,u). 

Therefore, max^o,!] sup feasible u> _ tj^^ = max I - e[0]1] Tf { x -\r,i,u) = max *e[o,i] r / (x,r,i,fi) ' Le " h 
is sufficient to consider only the u policy for the lemma. From Equation ([5]), it follows that, 

max — = max {A > | 3x G [0, 1] s.t. x € x cq (r, A)} . 

xe[o,i] Tf(x, t, 1, u) 

The result follows from Equation Q. ■ 
The following bound on Tf(x,r,n,u) will be critical in proving a sharp upper bound on A^ ax (r). 
Lemma 3.6: For any x € [0, 1], r > 0, n € N and u 6 U, we have that Tf(x,r,n,u) > n/A™ ax (r). 
Proof: For a given x G [0, 1], r > and u G U, we prove the result by induction on n. The 



statement holds true for n = 1 by Lemma 3.5 Assume that the result holds true for some n = k, i.e., 
T7(x, r, k, u) > /c/A™ ax (r). Given this, we shall prove that the statement holds true for n = k + 1. 

Without any loss of generality, assume that u does not let the server idle before assigning the first 
task. This is because if u lets the server idle initially until it reaches a state, say x < x, then one can 
alternately consider a modified (k + l)-task problem with initial and final server state x, and a modified 
control policy n mo d that does not idle the server before assigning the first task and under which the server 
states at the beginning of the service of tasks are the same as those under u. By following an argument 
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Fig. 2. Rearranging the time segments during the service cycle of a one-task static problem. 



similar to the one illustrated in Figure |2j one can then see that the time for this modified (k + l)-task 
problem, Tf(x, r,k + 1, u m0 d), is the same as Tt(x, r, k + 1, u). 

For i € {1, . . . , k + 1}, let x% and x[ denote the server states at the beginning and at the end of service 
of task i respectively under the policy u. As argued before, we assume without loss of generality that u 
is such that x\ = x, and hence, 

x\ > x. (6) 

For i G {1, . . . , k + 1}, let x' { ) = S(xi) denote the time required for the server state to go from 

Xi to x\ and for i 6 {1, ... , k}, let t_(x' i , Xi+i) = r(loga^ — logXj+i) denote the time required by the 
server to idle from x\ to Xj+i. With these notations, one can write that, 

k k 

T f (x,T,k + l,u) = + + t + (x k+1 ,x' k+1 ) + t-(x' k+1 ,x). (7) 

i=l i=l 

The rest of the proof is split among the following two cases. 
Case 1: x k+ i > x. We write t-(x' k+1 ,x) as 

t-(x' k+1 ,x) = Tlog(x / fc+1 /x A . +1 ) + T\og(x k+1 /x). (8) 

Therefore, from Equations Q and ([8]), we get that, 

k k-1 

T f (x,T,k + l,u) = ^2 l t + (x u x' i ) + ^2t-(x' i ,x i+ i) + t + (x k+1 ,x' k+1 ) + rlogx' fc+1 

i=l i=l 

-Tlogx fc+ i) +t-(x' k ,x k+1 ) + r(logx fc+ i -logz) 
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Since t-(x' k ,x k+ i) + r(logx fc+ i - log a;) = r{\ogx' k - logx fc+ i) + r(logx fc+ i - log a;) = r(logx' fc - 
logx) = t-(x' k ,x), we can write that, 

k fc-1 

T f (x,T,k + l,u) = ^2t+(x i ,x' i ) + '*rt-(x' i ,x i+ i) + t + (x k+1 ,x' k+1 ) 

i=i i=i (9) 

+ t-(x' k+1 ,x k+1 ) + t^(x' k ,x). 

Now, consider the fc-task static problem with initial and final server state x. Let u' denote the control policy 

under which the server states at the beginning of the tasks are X\,X2, ■ ■ ■ ,x k . Also, t + (x k+ i, x' k+l ) + 



t~(x' k+1 ,x k+ i) = Tf(xk+i,T,l,u), where u, as in the proof of Lemma 3.5 is the control policy for 
the one-task static problem that assigns the task to the server without any initial idling time. Combining 
these with Equation ([9]), one gets that, Tf(x,T, k + l,u) = Tj(x,t, k,u') + Tf(x k+ i,r, l,u). We have 
T f (x, t, k, u') > fc/A™ x (r) by the induction argument and T f (x k+1 ,r, 1, u) > 1/A™ x (r) by Lemma 
Therefore, T f (x, t, k + 1, u) > {k + 1)/A™ x (r). 

Case 2: x k+ \ < x. Let i := max{i G {1, . . . , k} \ x\ > x k. Xi + \ < x}. Since x' x > x by Equation ([6]), 
i is well defined. Therefore, 

i i— 1 



T f (x,T,k + l,u) =^2 l t + [x i ,x' i ) + ^2t-(x' i ,x i+ x) + t-(x~,x~ i+1 ) 



i=l i=l 

k+1 k 



(10) 



+ t +(xi,x' i )+ ^2 t-{x' i: x i+1 ) + t-(x' k+1 ,x). 

i=i+l i=i+l 

Splitting t-(x~,x~ i+1 ) as r(logx' — logx) + r(logx — logxj, 1 ), Equation ( |T0] > can be written as 



Tf(x,r, k + l,u) = Tf(x, r,i,ui) + Tf(x,r, k + l — i, u 2 ), (11) 

where u\ is the control policy for the i-task static problem with initial and final server state x, such that 
the server states at the beginning of the service of tasks are x\,...,x~ i , and u 2 is the control policy for 
the k + 1 — 2-task static problem with initial and final server state x such that the server states at the 
beginning of the service of the tasks are x~ i+l , . . . , x k+ \. Since both i and k + l — i are strictly less than 
+ we apply induction argument to both the terms on the right side of Equation (jTTJ to conclude that 
T f (x,T,k + l,u) > jdr^ + w\ = t^. ■ 

C. Upper Bound on Stabilizable Arrival Rate 

We now return to the dynamic problem, where we prove an upper bound on A^ ax (r). Trivially, 
Amax( T ) ^ 3T - ■ We next establish a sharper upper bound. First, we state a useful lemma. 



July 22, 2010 



DRAFT 



10 



Lemma 3.7: For any r > 0, xo G [0,1], no G N and A > A™ ax (r), if xth(T~) < 1 then there exist 
constants xl(t) and xu{t) satisfying < xl(t) < xjj{t) < 1 such that for any u G U under which 
the server states at the beginning of tasks do not lie in [xl{t),xjj{t)\ infinitely often, we have that 
lim sup t _> +00 n u (t,T, \,x ,n ) = +oo. 

Proof: We first define the constants xl(t) and xjj(t). For a given r > 0, let x m i n := 1 — e _5min / T 
denote a lower bound on the lowest possible server state immediately after the service of a task. Note 
that, for any r > and S m ; n > 0, x m - m > 0. For a given r > 0, define a map g : [0, 1] — > R U {+00} as: 

g(x) = 5 min + r log(x min /x). (12) 

Note that g is continuous, strictly decreasing with respect to x, and that g(0) = +00. Therefore, by 
continuity argument, there exists a x > such that g(x) > 1/A™ ax (r) for all x G [0, x). Define scj^t) := 
min{x m i n ,x}. It follows from the previous arguments that x^{t) > 0. Define the following quantities 

x Ul :=max{x G [0, 1] | S(x) = 1/A™ ax (r)} , 

(13) 



c U2 :=1 - (1 - x h )e TA "™ XM , x l2 := x U2 e . 

1 + %U\ 



x l (t) := mm{xi i: xi 2 }, x v (t) := max , } 

where we have dropped the dependency of x\ x , xn 2 , x Ul and x U2 on r for the sake of conciseness. 
We now infer relevant properties of the various quantities defined in Equation ( fT"3j ). Remark [374] implies 
that, if x th (r) < 1, then 5(1) > 1/A™ ax (r). This, combined with the fact that S min < 1/A™ ax (r) and 
that S(x) is continuous, implies that x Ul is well-defined and x Ul < 1. From the definition of x U2 , we 



have that x U2 < 1 and also x U2 > x^ since e tA =i (T) < 1. This, combined with earlier conclusion after 



Equation ( [T2| ) that x\ x > 0, implies that x U2 > 0. Therefore, from the definition of xi 2 , we have that, 
X[ 2 > and x\ 2 < x U2 . Combining these facts with the definitions of xl(t) and xjj{t), we have that, 

< x l {t) < x v {t) < 1. 

For the rest of the proof, we drop the dependency of xl and xjj on r. Consider a u such that the 
maximum task index for which the server state lies in [xl,X[/] is finite, say /. Let xi and x\ be the 
server states at the beginning of service of task i and the end of service of task i respectively. Consider 
a service cycle of a typical task for i > I. We now consider four cases depending on where X{ and Xi+\ 
belong, and in each case we show that the time between the beginning of successive tasks is strictly 
greater than 1/A™ ax (r), thereby establishing the lemma. 

• Xi G [0,xl) and Xi+% G [0, xl): The service time for task i, S(xi), is lower bounded by 5 m j n . By 
the definition of x m - m , x\ > x m - m and hence x' { > xl- Since Xi + \ is less than xl, the server has 
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to idle for time r \og{x' i / Xi + \), which is lower bounded by r\og{x m \ n / xl)- In summary, the total 
time between the service of successive tasks is lower bounded by 5 m i n + r log(x m ; n /x£), which is 
equal to g(xi) from Equation ( fT2] ). By the choice of xl, g(xL) is strictly greater than 1/A™ ax (r). 
Xi E (xu, 1] and x- l+ \ E 1]: The convexity of <S(x) along with Lemma [33] imply that S{x) > 



l/A™ ax (T) for all x E (x Ul , 1]. Since X[/ > x Ul from Equation ( fl3) , we have that <S(xj) > 1/A™ ax (r 



Therefore, the time spent between successive tasks is lower bounded by 1/A™ ax (r). 

Xj E [0, xl) and Xj + i E (xu, 1]: The fact that it takes at least 2/A™ ax (r) amount of service time on 

task i for the server to go from from x, to Xj + i follows from the definition of xi t and x U2 and their 



relation to xl and xu respectively, as stated in Equation ( fT3[ ). Therefore, the time spent between 
successive tasks is at least 2/A™ ax (r). 

Xi E (xu, 1] and Xj + i E [0, xj,): The fact that it takes at least 2/A™ ax (r) time for the server to 
idle from x\ to Xi+% follows from the definition of x/ 2 and x U2 and their relation to xl and xu 



respectively, as stated in Equation ( fT3| ). Therefore, the time spent between successive tasks is at 
least 2/A™ x (r). 

■ 

Theorem 3.8: For any r > 0, x E [0, 1], n Q 6 N, A > A™ ax (r) and u E W, if x th (r) < 1 then we 
have that lim sup t __ >+00 n u (t, r, A, xo, no) = +oo. 

Proof: Lemma |3.7| implies that there exist xl > and xu < 1 such that it suffices to consider set 
of task release control policies under which the server states at the beginning of service of tasks lie in 
[xl,xu] infinitely often. Consider one such control policy and let the sequence of indices of tasks for 

which the server state at the beginning of their service belongs to [xl, xu] be denoted as ii, Let 

Xi and ti be the server state and the time respectively at the beginning of the service of the z-th task. 
We have that Xi k E [xl,xu] for all k > 1. Define constants c\ and C2 as follows: 

a := -rlogx L , c 2 := -rlog(l - x v ). (14) 

Note that both c\ and c 2 are positive. For each k > 1, we now relate ti k — to the time for a 
related static problem. If Xi k > x^, then consider the (i^ — ii)-task static problem with initial and final 
server state x^. Then, for a control policy u' for this static problem under which the server states are 
x it ,x il+ i, ...,x it ,we have Tf(x il ,T,ik-ii,u') = t ik -t i± +r(logXj fc -logx^J. Therefore, t ik -t it = 
Tf(x il ,T,i k -ii,u')-T(logx ik -logx it ) > Tf(x il ,T,i k -ii,u')+Tlogx il > T f (x il ,T,i k -ii,u')-c 1 , 
where the last inequality follows from Equation ([14]). If Xi k < x^, then consider the (i k — i\ + m)-task 
static problem with initial and final server state Xi 1 and a control policy u" for this static problem such 
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that: the server states at the beginning of the service of first i k — i\ tasks are x^, aj^+i, . . . ,Xi k and m 
is the smallest number such that on servicing these m tasks without any idling after ifc-th task, one has 
%i k +m > An upper bound on the time for the static problem under u" is 

Tf(x il ,T,m + i k - h,u") <t ik - t it + rlog(l - ZfJ - rlog(l - x ik ) 

(15) 

+ 5 max - t log x h , 

where t^—t^ is the time between the beginning of service of tasks i\ and i k , r (log(l — XjJ — log(l — xi k )) 
is the time required for the server state to increase from Xi k to x^ under continuous usage, <S max 
is the upper bound on the overshoot time that could happen if the server is in middle of servicing 
a task while crossing state x^ and — rlogx^ is the upper bound on the time taken by the server 
to idle back to state x^ in case of the overshoot. Equation ( fl3] ) can be rewritten as U k — > 
T f (xjj ,r,m+i k -h,u")- 5 max + r log(l -x ik )+r log x h > T f (x h ,T,m + i k -i 1 ,u")- S max - c x - c 2 , 



where the last inequality follows from Equation ( 14 1. 



Combining these bounds on ti k — U 1 with lemma 3.6 we have that, for all k > 1, 



U k -t h >{ ^ { ' . (16) 
"ffir) 1 ~ 5max - c i - c 2 otherwise. 



With c = a + C2 + 5 max , Equation ( fT6| ) becomes 

*i» - > l k ~rh - c Vfc > 1. (17) 
1 ~ A™ x (r) 

For > 1, let n k be the queue length at the beginning of service of task i k . Then one can write 



n-k > n\ + X(ti k — tjj — (i k — i\) \/k > 1. This with Equation ( [T7| ) gives 



n k > ni - Ac + fa - tx) ( - 1 ) VA; > 1. (18) 



From Equation (_18i, we get that, for A > A™ ax (r), limfc^ +OC) = +oo. The theorem follows from the 
fact that limsup t _> +00 n u (t, r, A, x Q , n ) > lim fc ^ +00 n k . * 



Remark 3.9: (i) Theorem 3.8 implies that, for a given r > 0, if x t ^(r) < 1 then Aj^ ax (r) = A™ ax (r). 
(ii) If xthir) = 1, then one can show that, for any e > 0, there exists no stabilizing task release control 

policy for arrival rates greater than A^(r) + e, i.e., A^ ax (r) < A™ ax (r) + e. 
In the next section, we propose a simple task release control policy and prove that it is maximally 
stabilizable, i.e., for any A < A"q ax ("r), it ensures that the dynamical queue is stable. 
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u T p(t) 



IV. Control Policy and Lower Bound on the Stabilizable Arrival Rate 
Consider the following threshold policy: 

' ON if x(t) <x t h(r), 
OFF otherwise, 

where x t h(T) is defined in Equation ([4]). We now prove that this threshold policy is maximally stabilizing. 

Theorem 4.1: For any r > 0, xo 6 [0, 1], no £ N and A < A™ 3X (t), if x t h(T) < 1 then we have that 
lim sup^ +OC) n„ TP (t, r, A, x , no) < +oo. 

Proof: Let Xj and ^ be the server state and time instants respectively at the beginning of service 
of the i-th task. For brevity in notation, let n(t) be the queue length at time t. For any xo € [0, 1] 
and no G N, considering the possibility when xo > x t h(T) we have that n{t\) = max{0,no — 1, no — 
1 + LArlog(x /x t h)J}- We now prove that nfa) < n(h) + [(A - l/5 max ) (-rlog(l - x t h) + 5 max )] + 
[~— Arlogxth] for all i through the following two cases: 

• State 1: x\ = x t h- While n(ij) > 0, we have that Xi+i = x t h and t{ + \ — U = T/(x t h,T, 1,utp) = 
lMeq ax ( T )- Therefore, if A = A™ ax (r), then the arrival rate is same as the service rate and hence 
n(U) = n(ti) for all i. If X < A™ ax (r), then the service rate is greater than the arrival rate and 
hence there exists an i' > 1 such that n(tj) < n(fj_i) for all i < i' and n (ti> + 1/A™ ax (r)) = 
and hence Xj<+i < x t h- Thereafter, we appeal to the next case by resetting Xj- + i and U> + i as x\ 
and t\ respectively. Moreover, with these notations, n(t\) = 0. 

• State 2: x\ < x t h- While the queue length is non-zero, the server is never idle. The maximum amount 
of continuous service time required for the server state to cross x t h starting from any x\ < x t h is 
upper bounded by — rlog(l — x t h) + 5 max , where — r log(l — xth) is the time to go from x = 
to x = x t h when the server is continuously busy, and the 5 max term accounts for the fact that the 
server might be in middle of servicing a task when it reaches x = x t h and hence its server state will 
exceed x t h before it finishes the task. The maximum amount of time spent in such an overshoot is 
upper bounded by 5 max . Since l/5 max is the minimum rate at which the server will be servicing 
the tasks during this time, the increase in the queue length during this time is upper bounded 
by [(A — l/5 max ) (— rlog(l — xth) +5 max )]. Under the threshold policy, the possible overshoot 
above x t h will be followed by an idle time which is upper bounded by — rlogx t h, at the end of 
which the server state is x t h- The increase in queue length during this time is upper bounded by 
[~— At log Xth] • Therefore, the maximum queue length when the server state reaches x t h is upper 
bounded by n\ + |~(A - l/5 max ) (— r log(l - x t h) + 5 ma x)l + [— At logical- Thereafter, we appeal 
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to the earlier case by resetting x\ = xth an d n\ to be the number of outstanding tasks when the 
server state reaches x t ^. 

In summary, when the system is in State 1, if A = A™ ax (r), it stays there with constant queue length, 
else, the queue length monotonically decreases to zero at which point it enters State 2. When the system 
is in State 2, it stays in it for ever or eventually enters State 1 with bounded queue length. Collecting 
these facts, we arrive at the result. ■ 



Remark 4.2: (i) From Theorem 3.8 and Theorem 4.1 one can deduce that, for any r > 0, A* iax (7~) = 
A™ ax (r), and the threshold policy is a maximally stabilizing task release control policy. 

(ii) In general, for a given A' < A™ ax (r), the threshold policy with the threshold value set at any value 
in [Xg q (r, A'), x^ q (r, A')] would ensure stability of the queue for all values of A < A'. 

(iii) If Xth(T) = 1, then one can show that, given e > 0, there exists a 5(e) > such that the threshold 
policy with the threshold value set at 1 — 5(e) ensures stability of the queue for all arrival rates 
less than or equal to A™ ax (r) — e. 

V. Conclusions 

In this paper, we studied the stability problem of a dynamical queue whose service times are dependent 
on the state of a simple underlying dynamical system. The model for the service times is loosely inspired 
by the performance of a human operator in a persistent mission. We proposed a simple task release control 
policy for such a dynamical queue and proved that it ensures stability of the queue for the maximum 
possible arrival rate. In future, we plan to extend the analysis here to stochastic inter-arrival and service 
times and to general server dynamics. We also plan to design control policies for such queues that 
optimize other qualities of service such as average waiting time of an incoming task using, for example, 



the flexibility in choosing the threshold, as noted in part (ii) of Remark 4.2 
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